Human Body Measurement Using Thermographic Images

ABSTRACT

A medical image processing method performed by a computer, for measuring the spatial location of a point on the surface of a patient&#39;s body including: acquiring at least two two-dimensional image datasets, wherein each two-dimensional image dataset represents a two-dimensional image of at least a part of the surface which comprises the point, and wherein the two-dimensional images are taken from different and known viewing directions; determining the pixels in the two-dimensional image datasets which show the point on the surface of the body; and calculating the spatial location of the point from the locations of the determined pixels in the two-dimensional image datasets and the viewing directions of the two-dimensional images; wherein the two-dimensional images are thermographic images.

The present invention relates to: a medical data processing methodperformed by a computer; a computer program and system for measuring thespatial location of a point on the surface of a patient's body; andrelated methods, computer programs and systems.

Many medical applications require knowledge of the spatial location of apoint on the surface of a patient's body. The location is for examplegiven in two-dimensional or three-dimensional space and is for examplerepresented by two-dimensional or three-dimensional co-ordinates. Onepossible approach is to analyse a stereoscopic image of the patient'sbody in order to determine the spatial locations of one or more pointson the body's surface. A stereoscopic image is typically obtained bycapturing two or more two-dimensional images from different and knownviewing directions. In this document, the expression “different viewingdirections” encompasses both non-parallel viewing directions andparallel but non-identical viewing directions. A viewing direction istypically defined by an optical axis of a camera which is used tocapture a two-dimensional image.

The two-dimensional images are then analysed in order to find the pixelin each of the images which shows the same point on the surface of thebody. The spatial location of the point can be calculated from thelocations of the pixels in the two-dimensional images and the opticalproperties of the cameras. Examples of this approach are for exampledescribed in R. Hartley and A. Zisserman, “Multiple View Geometry inComputer Vision”, Cambridge University Press, 2^(nd) edition, 2003 andin X. Mei, X. Sun, M. Zhou, S. Jiao, H. Wang, and X. Zhang, “On buildingan accurate stereo matching system on graphics hardware”, GPUCV 2011.

An improved method for measuring the spatial location of a point on thesurface of a patient's body, a computer and a system are disclosed inthe independent claims. Advantageous embodiments, including othermethods which involve the method of the independent claim, are disclosedin the dependent claims.

One aspect of the present invention relates to a medical data processingmethod, performed by a computer, for measuring the spatial location of apoint on the surface of a patient's body. The method involves the stepof acquiring at least two two-dimensional image datasets, wherein eachtwo-dimensional image dataset represents a two-dimensional image of atleast a part of the surface which comprises the point, and wherein thetwo-dimensional images are taken from different and known viewingdirections. As explained above, different viewing directions can benon-parallel or parallel but non-identical viewing directions.

A two-dimensional image dataset is for example acquired from a datastorage or a camera. A camera comprises a lens system which images atleast the part of the surface which comprises the point onto a sensorwhich converts radiation emitted from the point into image data. Theproperties of the lens system, such as its focal length and anydistortions caused by the lens system, are preferably known to themethod. Optionally, the distortions are corrected within the camerabefore the method acquires the two-dimensional image dataset from thecamera. The output of the camera can be stored in the data storage andsubsequently acquired from the data storage.

The method may also involve the step of determining the pixels in thetwo-dimensional image datasets which show said point on the surface ofthe body. This means that one pixel is identified in eachtwo-dimensional image, wherein the identified pixels show the samepoint. The positions of the identified pixels in the respectivetwo-dimensional images depend on the location of the imaged point inspace. Pixels showing the same point are also referred to ascorresponding pixels. Many algorithms for determining correspondingpixels in a set of two-dimensional images are known within the field ofimage processing. Determining the pixels for example involvesdetermining the location of the pixels in the two-dimensional imagedatasets. If the two-dimensional image datasets are structured astwo-dimensional arrays of pixels, then the location of a pixel istypically given as a pair of co-ordinates which define a position in thearray.

The method may further involve the step of calculating the spatiallocation of the point from the locations of the determined pixels in thetwo-dimensional image datasets and the viewing directions of thetwo-dimensional images. Depending on how it is implemented, this stepcan also involve calculating the spatial location from the properties ofthe lens systems of the cameras used to capture the two-dimensionalimage datasets. Many algorithms for calculating the spatial location ofa point from a set of two-dimensional images are known within the fieldof image processing. One simple approach is to determine a straight linein space on which the point lies, for each of the two-dimensional imagedatasets, from the imaging properties of the respective camera and theposition of the respective pixel of the corresponding pixels in thetwo-dimensional image. The spatial location is then calculated as thepoint in space at which the straight lines corresponding to thetwo-dimensional image datasets intersect.

The spatial location of the point is for example given in a referenceco-ordinate system. This reference co-ordinate system can be a globalco-ordinate system, such as an Earth-relative co-ordinate system, or aco-ordinate system defined with respect to a room in which the camerasare located, or a co-ordinate system which is defined with respect tothe cameras.

In accordance with one aspect of the present invention, thetwo-dimensional images are thermographic images. In a thermographicimage, the pixels do not represent the colour of a point in the visiblespectrum, but rather a temperature or temperature distribution emittedfrom the point. Since each point emits a range of temperatures up to amaximum temperature, a pixel of a thermoscopic image can for examplerepresent this maximum temperature of the point. In this document, athermographic image preferably only represents the thermal radiationemitted from the surface of the patient's body in a non-visiblespectrum. A thermographic image can only be effectively presented to ahuman eye if frequency mapping is performed. This is often referred toas false-colour representation.

The advantage of using thermographic images to measure the spatiallocation of a point is that the spectrum of the thermal radiation isindependent of ambient conditions, such as illumination, and the opticalproperties of the surface, so even if a large area of the surface hasthe same optical properties in the visual spectrum, it may exhibit aparticular pattern in its thermal radiation, such that correspondingpoints in the set of two-dimensional image datasets can be found moreeasily and more reliably.

In known approaches using two-dimensional images in the visiblespectrum, the problem of uniform areas has been addressed by projectinga characteristic optical pattern onto the surface. Corresponding pointscan then be found using this known optical pattern. However, if thepatient's body, and therefore the surface, moves relative to theprojector emitting the optical pattern, a particular point of thispattern does not illuminate the same point on the surface over time. Bycontrast, the thermal pattern irradiated by a patient's body movestogether with the body relative to the cameras, such that a relativemovement between the cameras and the patient's body can be reliablydetermined.

In one embodiment, the spatial locations of a plurality of points on thesurface of the body are measured. This preferably involves utilising thesame two-dimensional image datasets for all points in the plurality ofpoints, but determining the pixels in the two-dimensional image datasetsand calculating the spatial location for each of the points in theplurality of points. The position of the surface of the patient's bodycan be determined from the spatial locations of a plurality of points,such as for example three or more points. In this document, the term“position” means the spatial location in up to three translationaldimensions and/or the rotational alignment in up to three rotationaldimensions.

In one embodiment, the method comprises the step of discarding pointswhich have spatial locations outside a predefined spatial range. In thisstep, a kind of spatial filtering is thus implemented. In this case,points outside a region of interest can be discarded. This can beadvantageous if the surface comprises two sub-surfaces which can moverelative to each other. It is then possible to discard points which donot lie on one of the sub-surfaces, such that the remaining points canbe more reliably registered to another dataset, because the discardedpoints, which can move relative to the remaining points, are ignored forthe purpose of registration.

The method can for example comprise the step of filtering thetwo-dimensional image datasets in order to discard pixels whichrepresent wavelengths outside a predefined wavelength range. In oneexample of this, pixels which represent wavelengths corresponding totemperatures below 20° C., 25° C. or 30° C. are discarded. This has theadvantage that pixels which do not belong to the surface of thepatient's body can for example be removed from the two-dimensional imagedatasets. This step is preferably performed before the step ofdetermining the pixels in the two-dimensional image datasets which showthe point on the surface of the body, such that the filtering stepreduces the computational effort involved in the determining step.

By combining spatial filtering and temperature selection, it is possibleto robustly decide whether a pixel is part of the surface of thepatient's body or not.

In one embodiment, the thermographic two-dimensional images representwavelengths between 8 μm and 14 μm. This range corresponds to typicaltemperatures for the surface of a patient's body. The thermographictwo-dimensional images preferably do not represent wavelengths in thenear infrared spectrum. The near infrared spectrum is typicallyunderstood to extend as far as wavelengths of 2.5 μm or 3 μm.

In one embodiment, the method comprises the step of assigning adescriptor to the point on the surface of the patient's body. Adescriptor is for example a value which is calculated from theproperties of the point and optionally also from the properties ofpoints in the vicinity of the point. A descriptor is typically used tounambiguously identify a point. In this document, a descriptor can alsobe a set of descriptors or descriptor values. A descriptor is forexample calculated from the two-dimensional image datasets, for examplefrom the properties of a pixel which represents the point on the surfaceand/or from the properties of pixels in the vicinity of this pixel.

One advantage of the descriptor is that it is basically invariant forsimilar viewing directions (such as viewing directions which form anangle of for example 1°, 2°, 5° or) 10° or over time (such as forexample for 1 second, 2 seconds, 5 seconds, 10 seconds, 15 seconds, 30seconds, 60 seconds or even longer). This means that the descriptor canadvantageously be used to identify pixels which show the same point onthe surface of the body in two-dimensional image datasets taken fromdifferent viewing directions and/or at different points in time.

The principles of calculating a descriptor are for example disclosed inM. Calonder, V. Lepetit, M. Özuysal, T. Trzcinski, C. Strecha, P. Fua,“BRIEF: Computing a Local Binary Descriptor Very Fast”, IEEETransactions on Pattern Analysis and Machine Intelligence, Volume 34,issue No. 07, Jul. 2012, pages 1281 to 1298, which is incorporated bythis reference.

In one aspect, the present invention also relates to a medical dataprocessing method for determining the alignment between a patient and athree-dimensional dataset which represents at least a part of a contourof the patient's body. The patient's body has a surface, and the shapeof the surface is the contour of the patient's body. The shape of thesurface can be represented by the locations of points on the surface,wherein the more points are measured, the more accurately the shape isrepresented.

This method involves the step of acquiring the three-dimensionaldataset. The three-dimensional dataset can for example represent thecontour by means of a set of points on the contour of the patient'sbody, such as a set of points created by measuring the spatial locationsof a plurality of points on the surface of a patient's body, asdescribed above. In another implementation, the three-dimensionaldataset can for example represent the contour by means of athree-dimensional image which shows the contour. The contour is forexample indicated by sufficiently different values of adjacent voxels ofthe three-dimensional image. The three-dimensional image can be obtainedby using any suitable imaging technique, such as for example magneticresonance imaging or computed tomography.

Typically, a dataset co-ordinate system is at least implicitly assignedto the three-dimensional dataset. This dataset co-ordinate system can bean internal co-ordinate system, such as a co-ordinate system with axesrepresenting the dimensions of a three-dimensional array of voxels, suchas for example a medical 3D image. The dataset co-ordinate system canalternatively be an external co-ordinate system, such as the referenceco-ordinate system. This is for example the case if thethree-dimensional dataset was acquired by using a medical imagingapparatus which was at a known position in the reference co-ordinatesystem. If the three-dimensional dataset represents a set of points,then the dataset co-ordinate system is for example a Cartesianco-ordinate system with a defined point of origin, and thethree-dimensional dataset represents the locations of the pointsrelative to this point of origin.

The three-dimensional dataset is typically obtained by measuring thecontour of the patient's body. At the time of measurement, the patient'sbody exhibits a particular spatial position. The three-dimensionaldataset therefore also exhibits a particular (virtual) spatial positionat this point. The spatial position of the three-dimensional dataset isfor example represented by the spatial position of the datasetco-ordinate system.

The method may also involve measuring the locations of a plurality ofpoints on the surface of the body, as described above. The time at whichthe locations of the plurality of points are measured is preferablylater than the time at which the three-dimensional dataset was created.

There are thus two representations of the surface of the patient's body,one provided by the three-dimensional dataset and one provided by thelocations of the plurality of points on the surface. If the shape of thepatient's body has not changed between the time the three-dimensionaldataset was created and the time the locations of the plurality ofpoints were measured, then the contour as represented by thethree-dimensional dataset will be exactly equal to the shape of thesurface as described by the locations of the plurality of points.

The method may also comprise the step of calculating alignmentinformation which represents a virtual relative position between thethree-dimensional dataset and the locations of the plurality of points,such that the measured locations of the plurality of points lie on thecontour of the body as represented by the three-dimensional dataset. Thealgorithm used for this calculation depends on the nature of thethree-dimensional dataset, for example on whether the three-dimensionaldataset is a three-dimensional image or a set of points. This is alsoknown as registration, and suitable algorithms are known to the personskilled in the art.

The virtual relative position can for example describe the relativeposition between the dataset co-ordinate system and a referenceco-ordinate system for which the contour represented by thethree-dimensional dataset matches the surface of the patient's body asdescribed by the spatial locations of the points. The difference betweenthis virtual relative position and the actual relative position betweenthe two co-ordinate systems represents the alignment, which can also bereferred to as a displacement, between the actual position of thesurface and the (virtual) position of the contour as represented by thethree-dimensional dataset.

The method can for example comprise the additional step of calculatingmovement control data from the alignment information. In one example,the movement control data describe how the patient's body has to bemoved within the reference co-ordinate system in order for the surfaceof the patient's body to match the contour of the patient's body asrepresented by the three-dimensional dataset. This is for example usefulfor moving the patient's body into a position which is identical to itsposition at the time the three-dimensional dataset was created. Oneexample application for this is radiotherapy, in which the patient'sbody has to be moved to a predetermined position relative to a linac(linear particle accelerator). If the three-dimensional dataset—which isassumed to have a particular position in the reference co-ordinatesystem, for example when it was created using a medical imagingapparatus which was at a known position in the reference co-ordinatesystem—represents the predetermined position of the patient's body inthe reference co-ordinate system, then the movement control data can beused to correctly align the patient's body in the reference co-ordinatesystem, for example relative to the linac.

Conversely, the movement control data can describe how thethree-dimensional dataset has to be positioned in the referenceco-ordinate system in order for it to represent the actual position ofthe patient's body in the reference co-ordinate system. This isadvantageous for image-guided procedures, in particular if medicalinstruments are tracked relative to the reference co-ordinate system andan image of the medical instrument is to be displayed together with thethree-dimensional dataset.

A further embodiment of the present invention also relates to a medicaldata processing method for verifying the position of at least a part ofa patient's body. This method involves the step of acquiring athree-dimensional dataset which represents at least a part of a contourof the part of the patient's body in a target position of the part ofthe patient's body. The target position is for example defined withrespect to the reference co-ordinate system and is for example a targetposition relative to a medical apparatus such as a linac.

The method may also comprise the step of measuring the locations of aplurality of points on the surface of the body, as described above.There are thus two representations of the patient's body, namely athree-dimensional dataset representing the contour of the patient's bodywhich describes the target position of the patient's body, and thelocations of the plurality of points which represent the actual positionof the patient's body.

The method may further comprise the step of determining whether or notthe locations of the plurality of points lie on the contour of the partof the patient's body as represented by the three-dimensional dataset.In other words, a determination is made as to whether or not the actualposition of the patient's body matches the target position of thepatient's body.

As in the method for virtually aligning a patient's body and thethree-dimensional dataset, the three-dimensional dataset can be athree-dimensional image dataset, such as a CT or an MR image dataset, ofat least a part of the patient's body which comprises the part of thecontour of the body.

As in the method for virtually aligning a patient's body and thethree-dimensional dataset, the three-dimensional dataset can be adataset which represents spatial locations of a plurality of points onthe part of the contour of the patient's body which were measured, asdescribed above.

It should be noted that the expressions “contour” and “surface” areoccasionally used in this document for brevity and are to be understoodto mean “at least a part of the contour” and “at least a part of thesurface”, respectively.

The present invention also relates to a medical data processing methodfor tracking at least a part of the surface of a patient's body. In thisdocument, the word “tracking” means determining the position atdifferent points in time.

This method comprises the step of measuring first locations of aplurality of points on the surface of the body, as explained above, at afirst point in time and measuring second locations of the plurality ofpoints, as explained above, at a second point in time which is laterthan the first point in time. This means that the locations of the sameplurality of points are measured at two different points in time.

The method also comprises the step of calculating a movement of at leastthe part of the surface of the body from the difference between thefirst locations and the second locations. This movement is preferablyrepresented by movement data. The movement is for example a translationin up to three translational dimensions and/or a rotation in up to threerotational dimensions.

The present invention also relates to a medical data processing methodfor determining a patient's vital signs. Vital signs measure basicfunctions of a body such as heartbeat or respiration.

This method comprises the step of measuring the location of at least onepoint, which is referred to as a tracked point, on the surface of thebody, as described above, at different points in time. It is assumedthat the movement of this point over time represents the basic functionof the body which is to be monitored.

The method also comprises the step of calculating the patient's vitalsigns from the measured locations of the tracked point over time. Thealgorithm used in this calculating step depends on the nature of thevital signs to be determined. Suitable algorithms for different kinds ofvital signs are known to the person skilled in the art.

The present invention also relates to a computer program embodied on anon-transitory computer-readable medium which, when running on acomputer or loaded onto a computer, causes the computer to perform anyone or more of the data processing methods described above.

The present invention also relates to a system for measuring the spatiallocation of a point on the surface of a patient's body, comprising: atleast two imaging units for recording two-dimensional thermographicimages of at least a part of the surface which comprises the point,wherein the two-dimensional thermographic images are represented bytwo-dimensional image datasets and the imaging units are arranged suchthat the two-dimensional images are taken from different and knownviewing directions; and a computer on which the program described aboveis stored and/or run.

The invention relates to the Applicant's product ExacTrac which is usedfor patient setup and monitoring in radiotherapy. ExacTrac currentlyutilises a marker-based IR tracking system in combination with astereoscopic X-ray system. While the X-ray system is very precise, itcan only be used to take snapshots of the position of the patient's bodyat well-defined points in time during the treatment. Otherwise, thepatient would be exposed to unjustifiable levels of radiation. Themarker-based IR system is used to monitor the patient in real time. Inorder to track the position of the patient's body, markers have to beattached either directly to the patient's body or to fixation deviceswhich fix the patient in a certain position.

Preparing each patient is time-consuming and can therefore reduce thenumber of patients who can be treated per day. Markers can become dirtyand may have to be serviced or even replaced. Fixation devices aregenerally considered uncomfortable for patients. Depending on theseverity of the patient's illness, treatments may have to be interruptedif for example the patient's very restricted freedom of movement causemental stress or even pain.

These problems can be successfully tackled by using a marker-lesstracking device as shown in the present invention. Using this system, itis possible to dispense with fixation devices and markers, and theposition of the patient's body can still be tracked precisely in realtime. Unlike other marker-less tracking devices, the present applicationallows for monitoring at much higher frame rates, since a far smallerquantity of data has to be computed as compared to typical structuredlight systems. There is also no need to synchronise separate systems,such as for example a projector and a camera.

Unlike a purely video-based approach, such as is currently used inacademic research, the present invention is independent of lightingconditions and/or reflections, which serves to stabilise the trackingresults. In hospitals, for example, the lights in the treatment room areoften dimmed in order to aid the patient in remaining calm duringtreatment.

Within the framework of the invention, computer program elements can beembodied by hardware and/or software (this includes firmware, residentsoftware, micro-code, etc.). Within the framework of the invention,computer program elements can take the form of a computer programproduct which can be embodied by a computer-usable, in particularcomputer-readable data storage medium comprising computer-usable, inparticular computer-readable program instructions, “code” or a “computerprogram” embodied in said data storage medium for use on or inconnection with the instruction-executing system. Such a system can be acomputer; a computer can be a data processing device comprising meansfor executing the computer program elements and/or the program inaccordance with the invention, in particular a data processing devicecomprising a digital processor (central processing unit or CPU) whichexecutes the computer program elements, and optionally a volatile memory(in particular a random access memory or RAM) for storing data used forand/or produced by executing the computer program elements. Within theframework of the present invention, a computer-usable, in particularcomputer-readable data storage medium can be any data storage mediumwhich can include, store, communicate, propagate or transport theprogram for use on or in connection with the instruction-executingsystem, apparatus or device. The computer-usable, in particularcomputer-readable data storage medium can for example be, but is notlimited to, an electronic, magnetic, optical, electromagnetic, infraredor semiconductor system, apparatus or device or a medium of propagationsuch as for example the Internet. The computer-usable orcomputer-readable data storage medium could even for example be paper oranother suitable medium onto which the program is printed, since theprogram could be electronically captured, for example by opticallyscanning the paper or other suitable medium, and then compiled,interpreted or otherwise processed in a suitable manner. The datastorage medium is preferably a non-volatile data storage medium. Thecomputer program product and any software and/or hardware described hereform the various means for performing the functions of the invention inthe example embodiments. The computer and/or data processing device canin particular include a guidance information device which includes meansfor outputting guidance information. The guidance information can beoutputted, for example to a user, visually by a visual indicating means(for example, a monitor and/or a lamp) and/or acoustically by anacoustic indicating means (for example, a loudspeaker and/or a digitalspeech output device) and/or tactilely by a tactile indicating means(for example, a vibrating element or a vibration element incorporatedinto an instrument). For the purpose of this document, a computer is atechnical computer which in particular comprises technical, inparticular tangible components, in particular mechanical and/orelectronic components. Any device mentioned as such in this document isa technical and in particular tangible device.

In the field of medicine, imaging methods (also called imagingmodalities and/or medical imaging modalities) are used to generate imagedata (for example, two-dimensional or three-dimensional image data) ofanatomical structures (such as soft tissues, bones, organs, etc.) of thehuman body. The term “medical imaging methods” is understood to mean(advantageously apparatus-based) imaging methods (so-called medicalimaging modalities and/or radiological imaging methods) such as forinstance computed tomography (CT) and cone beam computed tomography(CBCT, in particular volumetric CBCT), X-ray tomography, magneticresonance tomography (MRT or MRI), conventional X-ray, sonography and/orultrasound examinations, and positron emission tomography. The imagedata thus generated are also referred to as “medical imaging data”.Analytical devices in particular are used to generate the image data inapparatus-based imaging methods. The imaging methods are in particularused for medical diagnostics, to analyse the anatomical body in order togenerate images which are described by the image data. The imagingmethods are also in particular used to detect pathological changes inthe human body. However, some of the changes in the anatomicalstructure, in particular the pathological changes in the structures(tissue), may not be detectable and in particular may not be visible inthe images generated by the imaging methods. A tumour represents anexample of a change in an anatomical structure. If the tumour grows, itmay then be said to represent an expanded anatomical structure. Thisexpanded anatomical structure may not be detectable; in particular, onlya part of the expanded anatomical structure may be detectable.Primary/high-grade brain tumours are for example usually visible on MRIscans when contrast agents are used to infiltrate the tumour. MRI scansrepresent an example of an imaging method. In the case of MRI scans ofsuch brain tumours, the signal enhancement in the MRI images (due to thecontrast agents infiltrating the tumour) is considered to represent thesolid tumour mass. Thus, the tumour is detectable and in particulardiscernible in the image generated by the imaging method. In addition tothese tumours, referred to as “enhancing” tumours, it is thought thatapproximately 10% of brain tumours are not discernible on a scan and arein particular not visible to a user looking at the images generated bythe imaging method.

The method in accordance with the invention is a data processing method.The data processing method is preferably performed using technicalmeans, in particular a computer. The data processing method ispreferably constituted to be executed by or on a computer and inparticular is executed by or on the computer. In particular, all thesteps or merely some of the steps (i.e. less than the total number ofsteps) of the method in accordance with the invention can be executed bya computer. The computer for example comprises a processor and a memoryin order to process the data, in particular electronically and/oroptically. The calculating steps described are in particular performedby a computer. Determining steps or calculating steps are in particularsteps of determining data within the framework of the technical dataprocessing method, in particular within the framework of a program. Acomputer is any kind of data processing device, in particular electronicdata processing device. A computer can be a device which is generallythought of as such, for example desktop PCs, notebooks, netbooks, etc.,but can also be any programmable apparatus, such as for example a mobilephone or an embedded processor. A computer can in particular comprise asystem (network) of “sub-computers”, wherein each sub-computerrepresents a computer in its own right. The term “computer” includes acloud computer, in particular a cloud server. The term “cloud computer”includes a cloud computer system which in particular comprises a systemof at least one cloud computer and in particular a plurality ofoperatively interconnected cloud computers such as a server farm. Such acloud computer is preferably connected to a wide area network such asthe World Wide Web (WWW) and located in a so-called cloud of computerswhich are all connected to the World Wide Web. Such an infrastructure isused for “cloud computing”, which describes computation, software, dataaccess and storage services which do not require the end user to knowthe physical location and/or configuration of the computer delivering aspecific service. In particular, the term “cloud” is used in thisrespect as a metaphor for the Internet (or World Wide Web). Inparticular, the cloud provides computing infrastructure as a service(IaaS). The cloud computer can function as a virtual host for anoperating system and/or data processing application which is used toexecute the method of the invention. The cloud computer is for examplean elastic compute cloud (EC2) as provided by Amazon Web Services™. Acomputer may comprise interfaces in order to receive or output dataand/or perform an analogue-to-digital conversion. The data are forexample data which represent physical properties and/or which aregenerated from technical signals. The technical signals are typicallygenerated by means of (technical) detection devices (such as for exampledevices for detecting marker devices) and/or (technical) analyticaldevices (such as for example devices for performing imaging methods),wherein the technical signals are typically electrical or opticalsignals. The technical signals in particular represent the data receivedor outputted by the computer. The computer is preferably operativelycoupled to a display device which allows information outputted by thecomputer to be displayed, for example to a user. One example of adisplay device is an augmented reality device (also referred to asaugmented reality glasses) which can be used as “goggles” fornavigating. A specific example of such augmented reality glasses isGoogle Glass (a trademark of Google, Inc.). An augmented reality devicecan be used both to input information into the computer by userinteraction and to display information outputted by the computer.Another example of a display device would be a standard computer monitorcomprising for example a liquid crystal display which is operativelycoupled to the computer for receiving display control data from thecomputer in order to generate signals which are then used to displayimage information content on the display device. A specific embodimentof such a computer monitor is a digital light box. The monitor can alsobe the monitor of a portable, in particular handheld device such as asmart phone or personal digital assistant or digital media player.

The expression “acquiring data” encompasses (within the framework of adata processing method) the scenario in which the data are determined bythe data processing method or program. Determining data encompassesmeasuring physical quantities and transforming the measured values intodata, in particular digital data, and/or computing the data by means ofa computer and in particular within the framework of the method inaccordance with the invention. The meaning of “acquiring data” alsoencompasses the scenario in which the data are received or retrieved bythe data processing method or program, for example from another program,a previous method step or a data storage medium, in particular forfurther processing by the data processing method or program. Theexpression “acquiring data” can therefore also for example mean waitingto receive data and/or receiving the data. The received data can forexample be inputted via an interface. The expression “acquiring data”can also mean that the data processing method or program performs stepsin order to (actively) receive or retrieve the data from a data source,for instance a data storage medium (such as for example a ROM, RAM,database, hard drive, etc.), or via the interface (for instance, fromanother computer or a network). The data can be made “ready for use” byperforming an additional step before the acquiring step. In accordancewith this additional step, the data are generated in order to beacquired. The data are in particular detected or captured (for exampleby an analytical device). Alternatively or additionally, the data areinputted in accordance with the additional step, for instance viainterfaces. The data generated can in particular be inputted (forinstance into the computer). In accordance with the additional step(which precedes the acquiring step), the data can also be provided byperforming the additional step of storing the data in a data storagemedium (such as for example a ROM, RAM, CD and/or hard drive), such thatthey are ready for use within the framework of the method or program inaccordance with the invention. The step of “acquiring data” cantherefore also involve commanding a device to obtain and/or provide thedata to be acquired. In particular, the acquiring step does not involvean invasive step which would represent a substantial physicalinterference with the body, requiring professional medical expertise tobe carried out and entailing a substantial health risk even when carriedout with the required professional care and expertise. In particular,the step of acquiring data, in particular determining data, does notinvolve a surgical step and in particular does not involve a step oftreating a human or animal body using surgery or therapy. In order todistinguish the different data used by the present method, the data aredenoted (i.e. referred to) as “XY data” and the like and are defined interms of the information which they describe, which is then preferablyreferred to as “XY information” and the like.

As mentioned, the invention does not involve or in particular compriseor encompass an invasive step which would represent a substantialphysical interference with the body requiring professional medicalexpertise to be carried out and entailing a substantial health risk evenwhen carried out with the required professional care and expertise. Inparticular, the invention does not comprise a step of positioning amedical implant in order to fasten it to an anatomical structure or astep of fastening the medical implant to the anatomical structure or astep of preparing the anatomical structure for having the medicalimplant fastened to it. More particularly, the invention does notinvolve or in particular comprise or encompass any surgical ortherapeutic activity. The invention is instead directed as applicable topositioning a tool relative to the medical implant, which may be outsidethe patient's body. For this reason alone, no surgical or therapeuticactivity and in particular no surgical or therapeutic step isnecessitated or implied by carrying out the invention.

In the following, the invention is described with reference to theenclosed figures which represent preferred embodiments of the invention.The scope of the invention is not however limited to the specificfeatures disclosed in the figures.

FIG. 1 schematically shows a system for measuring the spatial locationof a point on the surface of a patient's body.

FIG. 2 schematically shows an example of virtually aligning a patientand a three-dimensional dataset.

FIG. 1 shows a system 1 for measuring the spatial location of a point onthe surface of the body of a patient P. The system 1 essentiallycomprises a computer 2 and a stereoscopic thermographic camera 3. Thecomputer 2 is connected to an input device 10, such as a keyboard or amouse, and to an output device 11 such as a monitor.

The stereoscopic thermographic camera 3 comprises two thermographicimaging units 4 a and 4 b. The imaging unit 4 a comprises a lens system5 a and a sensor 6 a. The imaging unit 4 b correspondingly comprises alens system 5 b and a sensor 6 b. The lens systems 5 a and 5 b guideincident thermal radiation onto the sensors 6 a and 6 b, respectively,wherein each of the sensors 6 a and 6 b creates a two-dimensionalthermographic image which preferably represents wavelengths of between 8μm and 14 μm. The lens systems 5 a and 5 b have characteristic axessimilar to the optical axis of a camera which captures an image in thevisible spectrum. As can be seen from FIG. 1, the two imaging units 4 aand 4 b have different characteristic axes and therefore differentviewing directions. The characteristic axes are shown as dashed lines inFIG. 1.

Thermal radiation emitted from a point on the body is guided ontocorresponding pixels of the sensors 6 a and 6 b in accordance with thespatial location of the point on the surface of the patient's body andthe characteristics of the lens systems 5 a and 5 b.

In the present example, the sensors 6 a and 6 b are two-dimensionalarrays of sensor cells which convert incident thermal radiation into avoltage which corresponds to the temperature of the corresponding pointon the surface of the patient's body. The temperature is typicallyderived from the wavelength of the maximum within the spectrum of theincident infrared radiation.

The computer 2 comprises a central processing unit 7, a memory unit 8and an interface 9. The memory unit 8 stores program data and/or workingdata, such as the image datasets acquired from the stereoscopic camera3. The computer is connected to the input device 10, the output device11 and/or the stereoscopic camera 3 via an interface 9.

The computer 2 acquires the two two-dimensional image datasets, whichwere captured using the sensors 6 a and 6 b, from the stereoscopiccamera 3. The computer 2 is provided with the properties of thestereoscopic camera 3, such that for each pixel in each of thetwo-dimensional thermographic image datasets, the computer 2 knows or isable to calculate the line on which points imaged by said pixel arelocated.

The computer 2 determines the pixels in the two two-dimensional thermalimages which capture the thermal radiation emitted from the same pointon the surface of the patient's body. The pixels are for exampledetermined by means of a descriptor which describes the thermalsignature of the point and the area surrounding this point, such thatthe descriptor is characteristic of this point.

For each of the two-dimensional cameras 4 a and 4 b, the computer 2 usesthe position of the determined pixel in the two-dimensionalthermographic image and the properties of the lens system 5 a or 5 b,respectively, to determine the line in space on which the point on thesurface of the patient's body lies. These lines are shown as solid linesin FIG. 1. The computer 2 then calculates the point in space at whichthe two lines intersect each other. The location of this point is thelocation of the point on the surface of the patient's body.

If the computer 2 measures the spatial locations of a plurality ofpoints on the surface of the patient's body, a set of points is obtainedwhich represents the shape of the surface of the patient's body.

One advantage of using thermographic images rather than images in thevisible spectrum is that the thermal signature of the body isindependent of the optical characteristics of the surface of thepatient's body and/or the characteristics of the light which is emittedonto the patient's body.

FIG. 2 shows the principle of aligning the patient P and athree-dimensional dataset which represents at least a part of a contourof the body of the patient P. In this embodiment, the three-dimensionaldataset represents a medical three-dimensional image such as an MR or CTimage. The three-dimensional dataset comprises a three-dimensional arrayof voxels. The contour of the patient's body can be derived from thevalues of the voxels.

FIG. 2 shows the body of the patient P in a reference co-ordinate systemC_(R). The computer 2 then measures the spatial locations of a pluralityof points on the surface of the patient's body in the referenceco-ordinate system C_(R). The position of the stereoscopic camera 3 inthe reference co-ordinate system C_(R) is known, such that the computer2 can measure the spatial locations of the points in this referenceco-ordinate system C_(R).

In order to make the example easier to understand, FIG. 2 shows thesurface of the body of the patient P rather than the points on thesurface of the patient's body. FIG. 2 also shows the contour of thepatient's body as represented by the three-dimensional dataset DS. Adataset co-ordinate system C_(DS) is assigned to the three-dimensionaldataset DS. The position of the contour as shown in FIG. 2 correspondsto the position of the patient's body when the three-dimensional datasetDS was captured.

The computer 2 calculates alignment information which represents arelative position between the three-dimensional dataset DS and thelocations of the plurality of points on the surface of the patient'sbody, such that the locations of the plurality of points lie on thecontour of the patient's body as represented by the three-dimensionaldataset DS. The alignment information describes the position of thedataset co-ordinate system C_(DS) in the reference co-ordinate systemC_(R), such that the contour represented by the three-dimensionaldataset DS matches the surface of the patient's body as represented bythe measured spatial locations of the points on the surface of thepatient's body. This position of the dataset co-ordinate system C_(DS)is shown as the co-ordinate system C_(DS)′ in FIG. 2.

If the dataset co-ordinate system C_(DS) has a particular initialposition in the reference co-ordinate system C_(R), for example if thethree-dimensional dataset DS was created by a medical imaging devicewhich was at a known position in the reference co-ordinate system C_(R),then a transformation T can be calculated in order to align thethree-dimensional dataset DS with the actual position of the patient'sbody, i.e. to register the dataset DS to the patient P. Once thethree-dimensional dataset DS is aligned with the patient's body, thethree-dimensional dataset DS can be displayed on the display device 11,for example together with an image of a medical instrument which istracked in the reference co-ordinate system C_(R), for example by aknown medical tracking system.

Conversely, if the initial position of the dataset co-ordinate systemC_(DS) corresponds to a target position of the patient's body in thereference co-ordinate system C_(R), then the inverse of thetransformation T describes how the actual position of the patient's bodyin the reference co-ordinate system C_(R) needs to be changed so as tomatch the target position. One application of this is in order to movethe patient P into the position which is equal to the position in whichthe three-dimensional dataset DS was created.

1. A system including memory and one or more processors operable toexecute instructions stored in the memory, comprising instructions to:acquire a three-dimensional image dataset; acquire at least twotwo-dimensional image datasets, wherein each two-dimensional imagedataset represents a two-dimensional thermographic image of at least apart of a surface of a patient's body which comprises a plurality ofpoints and wherein the two-dimensional thermographic images are takenfrom different and known viewing directions; determine, for each pointin the plurality of points, the pixels in the two-dimensional imagedatasets which show said point on the surface of the patient's body; andcalculate, for each point in the plurality of points, the spatiallocation of the point from the locations of the corresponding determinedpixels in the two-dimensional image datasets and the viewing directionsof the two-dimensional thermographic images; and calculate alignmentinformation which represents a virtual relative position between thethree-dimensional image dataset and the locations of the plurality ofpoints, such that the measured locations of the plurality of points lieon a contour of the body as represented by the three-dimensional imagedataset.
 2. A computer implemented method for tracking at least a partof a surface of a patient's body, comprising: measuring, by the one ormore processors, at a first point in time, first locations of aplurality of points on the surface of the patient's body by acquiring atleast two two-dimensional image datasets, wherein each two-dimensionalimage dataset represents a thermographic two-dimensional image of atleast a part of the surface of the patient's body which comprises theplurality of points, and wherein the two-dimensional thermographicimages are taken from different and known viewing directions; assigning,by the one or more processors, a descriptor to at least one of thepoints on the surface of the patient's body; determining, by the one ormore processors, using at least the assigned descriptors for theplurality of points on the surface of the patient's body, the pixels inthe two-dimensional image datasets which show said plurality of pointson the surface of the body; and calculating, by the one or moreprocessors, the spatial location of the plurality of points from thecorresponding locations of the determined pixels in the two-dimensionalimage datasets and the viewing directions of the two-dimensionalthermographic images; measuring, by the one or more processors, at asecond point in time which is later than the first point in time, secondlocations of the plurality of points in the same manner as the firstlocations; and calculating, by the one or more processors, a movement ofat least the part of the surface of the body from the difference betweenthe first locations and the second locations.
 3. The method of claim 2,further comprising discarding, by the one or more processors, at leastone of the plurality of points which have spatial locations outside apredefined spatial range.
 4. The method of claim 2, further comprisingfiltering, by one or more processors, the two-dimensional image datasetsin order to discard pixels which represent wavelengths outside apredefined wavelength range.
 5. The method of claim 2 wherein thethermographic two-dimensional images represent wavelengths between 8 μmand 14 μm.
 6. A non-transitory computer-readable program storage mediumcomprising instructions which, when executed by at least one processor,causes at least one processor to track at least a part of a surface of abody of a patient, the at least one processor executing the steps of:measuring, at a first point in time, first locations of a plurality ofpoints on the surface of the body of the patient by acquiring at leasttwo two-dimensional image datasets, wherein each two-dimensional imagedataset represents a thermographic two-dimensional image of at least thepart of the surface of the patient's body which comprises the pluralityof points, and wherein the two-dimensional thermographic images aretaken from different and known viewing directions; assigning adescriptor to at least one point in the plurality of points, thedescriptor for each of the at least one point including a calculatedvalue from the properties of the at least one point and also theproperties of further points in the vicinity of the point; determiningthe pixels in the two-dimensional image datasets which show saidplurality of points on the surface of the body of the patient using atleast one of the assigned descriptors; calculating the spatial locationof the plurality of points from the corresponding locations of thedetermined pixels in the two-dimensional image datasets and the viewingdirections of the two-dimensional thermographic images; measuring, at asecond point in time which is later than the first point in time, secondlocations of the plurality of points in the same manner as the firstlocations; and calculating a movement of at least the part of thesurface of the body of the patient from the difference between the firstlocations and the second locations.